Are Teacher-Level Value-Added Estimates Biased? An Experimental Validation of Non-Experimental Estimates

For more than three decades, analyses of non-experimental data have reported considerable heterogeneity in teacher impacts on student achievement. We use data from a random-assignment experiment in Los Angeles Unified School District to test the validity of various non-experimental specifications. To do so, we first generated estimates for a group of teachers during a pre-experimental period and then tested the validity of those teacherlevel estimates in predicting student achievement following random assignment. First, although the validation exercise revealed a slight advantage for a specification including prior student test scores and mean classroom characteristics as covariates (both in terms of bias and mean-squared error), several alternative non-experimental specifications were able to predict experimental outcomes. Second, although there is greater signal variance in teacher effects on math performance than on English language arts, the impacts on math performance seem to fade-out more rapidly. Specifically, half of the impact on math performance faded after one year and another half disappeared the following year; the impacts on English language arts are smaller in magnitude but more long-lived. Third, the same pattern of substantial fade-out in math (and less fade-out in English language arts) is apparent in the non-experimental data.